Efficient audio signal processing in the sub-band regime

ABSTRACT

A signal processing system enhances an audio signal. The audio signal is divided into audio sub-band signals. Some audio sub-band signals are excised. Other audio sub-band signals are processed to obtain enhanced audio sub-band signals. At least a portion of the excised audio sub-band signals are reconstructed. The reconstructed audio sub-band signals are synthesized with the enhanced audio sub-band signals to form an enhanced audio signal.

PRIORITY CLAIM

This application is a divisional application of U.S. application Ser.No. 12/241,788, filed Sep. 30, 2008, which claims the benefit ofEuropean Patent Application No. 07019281.0, filed Oct. 1, 2007. Both ofthese applications are incorporated by reference in their respectiveentireties.

BACKGROUND OF THE INVENTION

1. Technical Field

This application relates to signal processing and, more particularly, toprocessing audio signals in a sub-band regime.

2. Related Art

Audio communication systems may operate in noisy environments. Noise mayinterfere with some communication systems, such as voice recognitionsystems and hands-free voice communication systems. When a voicerecognition system operates in a noisy environment, the noise mayinterfere with the ability of the voice recognition computer torecognize a user's voice commands Hands-free voice communication systemsmay also be susceptible to background noise and echo. Echo may occurwhen a system microphone picks up an audio signal played from a systemloudspeaker.

To increase the quality of these communications, audio communicationsystems may process the audio signals to remove noise and/or echocomponents. This type of processing may be computationally complex. Forexample, memory demand and computation time may be relatively high forthese processes. Therefore, a need exists for a more efficient way toprocess audio signals.

SUMMARY

A signal processing system enhances an audio signal. The audio signal isdivided into audio sub-band signals. Some audio sub-band signals areexcised. Other audio sub-band signals are processed to obtain enhancedaudio sub-band signals. At least a portion of the excised audio sub-bandsignals are reconstructed. The reconstructed audio sub-band signals aresynthesized with the enhanced audio sub-band signals to form an enhancedaudio signal.

Other systems, methods, features, and advantages will be, or willbecome, apparent to one with skill in the art upon examination of thefollowing figures and detailed description. It is intended that all suchadditional systems, methods, features and advantages be included withinthis description, be within the scope of the invention, and be protectedby the following claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The system may be better understood with reference to the followingdrawings and description. The components in the figures are notnecessarily to scale, emphasis instead being placed upon illustratingthe principles of the invention. Moreover, in the figures, likereferenced numerals designate corresponding parts throughout thedifferent views.

FIG. 1 shows a signal processing system.

FIG. 2 is one implementation of the signal processing system of FIG. 1.

FIG. 3 is a process that enhances an audio signal.

FIG. 4 is a process that uses a reference signal to enhance an audiosignal.

FIG. 5 is a process that reconstructs excised sub-band signals.

FIG. 6 is a process that compensates for echo in a microphone signal.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A signal processing system may enhance an audio signal. The system mayprocess the audio signal in a sub-band regime by dividing the audiosignal into multiple sub-band signals. A portion of the sub-band signalsmay be excised to increase signal processing efficiency, and a remainingportion of the sub-band signals may be processed to obtain an enhancedaudio signal.

FIG. 1 shows a signal processing system 102 in communication with anaudio communication system 104. The audio communication system 104 maybe a voice recognition system, hands-free voice communication system, orother audio system. An audio detection device 106 may interface with theaudio communication system 104. The audio detection device 106 mayinclude one or more microphones or other devices that detect audiosignals and transmit the detected signals to the audio communicationsystem 104 for processing. The audio communication system 104 may alsointerface with one or more loudspeakers 108. The loudspeakers 108 mayreceive audio signals from the audio communication system 104 and makethose signals audible for users in a vicinity of the loudspeakers 108.

In some implementations, the audio communication system 104 mayinterface with a communication network 110. The audio communicationsystem 104 may transmit audio signals across the communication network110 to one or more other communication systems. Also, the audiocommunication system 104 may receive audio signals from othercommunication systems through the communication network 110. In oneimplementation, a user of the audio communication system 104 mayparticipate in a voice conversation with a remote party through thecommunication network 110. The audio detection device 106 may detect theuser's speech, and the loudspeakers 108 may transmit speech receivedfrom the remote party.

The audio communication system 104 may operate in a noisy environment.The noise may include background noise, echo, or other interference.Echo may occur in the communication system 104 when the audio detectiondevice 106 picks up an audio signal transmitted from the loudspeakers108. The audio communication system 104 may use the signal processingsystem 102 to attenuate noise and obtain an enhanced audio signal.

FIG. 2 illustrates one implementation of the signal processing system102. In FIG. 2, the signal processing system 102 may enhance an audiosignal y(n). The audio signal y(n) may be part of a telephoneconversation between a remote party and a near party. The near party mayuse a hands-free set that includes a loudspeaker (e.g., the loudspeaker108 of FIG. 1) and a device that converts speech into an analog ordigital signal (e.g., the audio detection device 106 of FIG. 1). Acommunication system at the near side (e.g., the communication system104 of FIG. 1) may receive a signal x(n) from the remote party. Thesignal x(n) may be speech from the remote party. The communicationsystem may convert the signal x(n) into an audible range through aloudspeaker.

The near party and the loudspeaker may be contained within a room (e.g.,a vehicle compartment or other space). The room may be part of aloudspeaker-room-microphone (LRM) system 202. The LRM system 202 may becharacterized by an impulse response h(n). Although the microphone ofthe LRM system 202 may be tuned to detect a speech signal s(n) of thenear side speaker, the microphone may also detect background noise b(n)and an echo contribution d(n) caused by the loudspeaker output.Therefore, the audio signal generated by the microphone may berepresented as y(n)=s(n)+b(n)+d(n).

The signal processing system 102 enhances the audio signal y(n) byprocessing the audio signal y(n) in a sub-band regime. To process theaudio signal y(n) in the sub-band regime, the audio signal y(n) isfiltered by an analysis filter bank g_(μ,ana) 204 to obtain the audiosub-band signals y_(sb)(n). The analysis filter bank 204 may includelow-pass, band pass, and/or high-pass filters. In some implementations,the analysis filter bank 204 may be formed from one or more DiscreteFourier Transformation (DFT) filters, Discrete Cosine Transformation(DCT) filters, or Fast Fourier Transformation (FFT) filters. Theanalysis filter bank 204 may comprise a Hann or Hamming window. Theanalysis filter bank 204 divides the audio signal y(n) into M audiosub-band signals y_(sb)(n). M represents the order of the DFT, DCT orFFT filters, for example, or the channel number of the analysis filterbank 204, in general.

After the audio signal y(n) is divided into multiple audio sub-bandsignals y_(sb)(n), the audio sub-band signals y_(sb)(n) may be passed toa filter 206. The filter 206 may serve to excise a portion of the audiosub-band signals y_(sb)(n). The filter 206 may excise a subset of theaudio sub-band signals y_(sb)(n) leaving a remaining subset of audiosub-band signals y_(sb,g). The filter 206 may excise a predeterminednumber of the audio sub-band signals y_(sb)(n), such as every otheraudio sub-band. In the implementation of FIG. 2, the filter 206 mayexcise each of the audio sub-band signals y_(sb)(n) with an odd indexnumber. Therefore, the remaining audio sub-band signals y_(sb,g) may bethe audio sub-band signals y_(sb)(n) that have an even index number.

The remaining audio sub-band signals y_(sb,g) may be filtered to enhancesignal quality. In one implementation, a Wiener filter may attenuatenoise components of the remaining audio sub-band signals y_(sb,g). Inthe implementation of FIG. 2, the remaining audio sub-band signalsy_(sb,g) are filtered by an echo compensation filter 208 that may berepresented by the following equation:ĥ _(μ)(n)=[ĥ _(μ,0)(n),ĥ _(μ,1)(n), . . . ĥ _(μ,N-1)(n)]^(T).The echo compensation filter 208 may have a length N (e.g., the numberof filter coefficients for each sub-band μ) for modeling the impulseresponse of the LRM system 202. The echo compensation filter 208 may bean infinite impulse response filter (IIR), adaptable finite impulseresponse filter (FIR), or other filter to compensate for echo effects.In some applications, about 256/r to about 1000/r filter coefficientsmay be used, where r denotes the factor of down-sampling of the sub-bandsignals. In other implementations, a different number of filtercoefficients may be used.

Some adaptation methods for the echo compensation filter 208 may beiterative methods (e.g., in full band):ĥ(n+1)=ĥ(n)+Δĥ(n).

In one implementation, the adaptation method for the echo compensationfilter 208 may be the normalized least mean square (NLMS) algorithm:

${\hat{h}\left( {n + 1} \right)} = {{\hat{h}(n)} + {{\kappa(n)}\;{\frac{x(n){e(n)}}{{{x(n)}}^{2}}.}}}$

The vector of the reference signal may be represented by:x(n)=[x(n),x(n−1), . . . ,x(n−N+1)]^(T).

The error signal e(n) represents the difference of the audio signal(e.g., the signal detected by the microphone) and the output of the echocompensation filter 208. The error signal e(n) may be represented by:e(n)=y(n)−{circumflex over (d)}(n)=y(n)−ĥ ^(T)(n)x(n).

The corrector step is adjusted by means of the real number κ.Accordingly, in the sub-band regime the normalized least mean square(NLMS) algorithm may be:

${{\hat{h}}_{sb}\left( {n + 1} \right)} = {{{\hat{h}}_{sb}(n)} + {{\kappa_{sb}(n)}\;{\frac{{x_{sb}(n)}{e_{sb}^{*}(n)}}{{{x_{sb}(n)}}^{2}}.}}}$The asterisk denotes the complex conjugate and κ_(sb)(n) adjusts thecorrector step. The vector of the reference signal may be representedby:x _(sb)(n)=[x _(sb)(n),x _(sb)(n−1), . . . ,x _(sb)(n−N+1)]^(T) ande _(sb)(n)=y _(sb)(n)−{circumflex over (d)} _(sb)(n)=y _(sb)(n)−ĥ _(sb)^(H)(n)x _(sb)(n).The upper index H denotes the Hermitian adjugate.

Because the filter 206 excises a portion of the audio sub-band signalsy_(sb)(n), the echo compensation filter 208 may operate on the sub-bandsthat correspond to the remaining audio sub-band signals y_(sb,g). If thefilter 206 excised the odd sub-bands and passed the even sub-bands, thenthe echo compensation filter 208 may operate on only the even sub-bandsas well. Therefore, the echo compensation filter 208 may savecomputational resources and time by not echo compensating the oddsub-bands.

To echo compensate only the sub-bands that are passed by the analysisfilter bank 206, the signal processing system 102 may process thereference signal x(n) in a manner that is similar to the processingapplied to the detected signal y(n). In one implementation, thereference signal x(n) may be passed through an analysis filter bank 210to obtain reference sub-band signals x_(sb)(n). Specifically, theanalysis filter bank 210 divides the reference signal x(n) into multiplereference sub-band signals x_(sb)(n). The analysis filter bank 210 maybe substantially similar or identical to the analysis filter bank 204used for dividing the detected signal y(n) into sub-bands. In oneimplementation, the analysis filter bank 210 may comprise a Hann orHamming window.

After the reference signal x(n) is divided into multiple referencesub-band signals x_(sb)(n), the reference sub-band signals x_(sb)(n) arepassed to a filter 212. The filter 212 serves to excise a portion of thereference sub-band signals x_(sb)(n). The filter 212 may excise a subsetof the reference sub-band signals x_(sb)(n) leaving a remaining subsetof reference sub-band signals x_(sb,g). In one implementation, theremaining reference sub-band signals x_(sb,g) are equal in number to theremaining audio sub-band signals y_(sb,g). The filter 212 may excise thereference sub-band signals x_(sb)(n) that correspond to the audiosub-band signals y_(sb)(n) that were excised by the filter 204. In theimplementation of FIG. 2, the filter 206 may excise each of the audiosub-band signals y_(sb)(n) with an odd index number. Therefore, thefilter 212 may excise each of the reference sub-band signals x_(sb)(n)with an odd index number. The remaining reference sub-band signalsx_(sb,g) may be the reference sub-band signals y_(sb)(n) that have aneven index number. The remaining reference sub-band signals x_(sb,g) maynext be passed to the echo compensation filter 208 where error signalse_(sb,g)(n) are obtained. The error signals e_(sb,g)(n) represent echocompensated audio sub-band signals.

In one implementation, the detected sub-band signals and the referencesub-band signals may down-sampled by a factor r. The audio sub-bandsignals y_(sb)(n) and the reference sub-band signals x_(sb,g)(n) may bedown-sampled with respect to the audio signal y(n) and the referencesignal x(n), respectively, by the same down-sampling factor r. If, e.g.,a Hann window is used for the analysis filter banks 204 and 210, thenthe length of the analysis filters may be equal to the number ofsub-bands M. For a typical processing of the analysis and the synthesisfilter bank by Discrete Fourier Transformation (DFT), for example, thelengths of the analysis and the synthesis filter banks may be the sameand equal to the number of sub-bands M. In one implementation, adown-sampling factor of r=M/4 may be used, which allows for goodre-synthesis of the audio sub-band signals. The spectra of thedown-sampled reference sub-band signals may be represented by:

${X_{\mu}\left( {\mathbb{e}}^{j\;\Omega} \right)} = {\overset{r - 1}{\sum\limits_{m = 0}}{{X\left( {\mathbb{e}}^{{j{\lbrack{\frac{\Omega}{r} - \frac{2\pi}{r}}\rbrack}}m} \right)}{G_{\mu,{ana}}\left( {\mathbb{e}}^{j{\lbrack{\frac{\Omega}{r} - {\frac{2\pi}{r}m}}\rbrack}} \right)}\mspace{14mu}{for}\mspace{14mu}{each}\mspace{14mu}{sub}\text{-}{band}\mspace{14mu}{\mu.}}}$

By down-sampling the audio sub-band signals with respect to the audiosignal detected by a microphone (e.g., with a sampling rate of about 8kHz) the computational load may be reduced. An increase in the rate rmay result in a reduction of the computational load. Due to thefinite-slope filter flanks, r=M may be an upper limit for the samplingrate r, where M represents the number of sub-bands (e.g., the number ofchannels of the analysis filter banks 204 and 210).

In the implementation shown in FIG. 2, the error signals e_(sb,g)(n) arefurther processed for noise reduction and reduction of residual echoesby a post-filter 214. The post-filter 214 may be a Wiener filter. Theresidual echoes may be due to imperfect adaptation of the echocompensation filter 208. The filter characteristics of the post-filter214 may be adapted based on the estimated auto power density of theerror signals e_(sb,g)(n) and the perturbation that is still present inthe error signals e_(sb,g)(n) (i.e., the echo compensated audio sub-bandsignals) in form of background noise and residual echoes.

The enhanced sub-band signals ŝ_(sb,g) (n) may be transferred from thepost-filter 214 to a processor 216. The processor 216 serves toreconstruct at least some of the excised audio sub-band signals.Specifically, the processor 216 may reconstruct sub-band signals for theaudio sub-band signals that were excised by the filter 206. In theimplementation of FIG. 2, the filter 206 excised the audio sub-bandsignals with odd index numbers. Therefore, the processor 216 generatesaudio sub-band signals to replace the original audio sub-band signalsthat had odd index numbers. The processor 216 may use the remainingaudio sub-band signals to reconstruct the excised audio sub-bandsignals.

In one implementation, reconstruction may be based on one previous andone following sub-band vector. From the vector of the audio signal y(n),where n is the discrete time index, a vector of some length M+2r (wherer denotes the factor of down-sampling of the sub-band signals) isextracted:y(n)=[y(n+r),y(n+r−1), . . . ,y(n−M−r+1)]^(T)where the upper index T denotes the transposition operation. Windowingmay be performed by:

$F = \begin{bmatrix}g_{0} & 0 & 0 & \ldots & 0 \\0 & g_{1} & 0 & \ldots & 0 \\0 & 0 & g_{2} & \ldots & 0 \\\vdots & \vdots & \vdots & \ddots & \vdots \\0 & 0 & 0 & \ldots & g_{M - 1}\end{bmatrix}$where the diagonal coefficients g₀, . . . , g_(M-1) are the coefficientsof the 0^(th) prototype filter (e.g., a Hann window) of the analysisfilter bank that is given by:g _(μ,ana) =[g _(μ,0,ana) ,g _(μ,1,ana) , . . . ,g _(μ,N) _(ana)_(-1,ana)]^(T).

The analysis filter banks may operate in the frequency (Ω) domain andthe frequency response of a prototype low-pass filter may be given by:

${G_{0,{ana},{ideal}}\left( {\mathbb{e}}^{j\Omega} \right)} = \left\{ \begin{matrix}{1,{{{for}\mspace{14mu}{\Omega }} \leq \frac{2\pi}{M}},} \\{{any},{{{for}\mspace{14mu}\frac{2\pi}{M}} < {\Omega } < \frac{2\pi}{r}},} \\{0,{{{for}\mspace{14mu}{\Omega }} \geq \frac{2\pi}{r}}}\end{matrix} \right.$

The other filters (sub-band index μ=1, . . . , M−1) may be obtained byfrequency shifting. After supplementation of the window matrix F withM×r zeros (zero padding) on the left-hand and right-hand sidesF₀=[0_(M×r) F 0_(M×r)], a windowed signal portion of the length M may beobtained by F₀ y(n). After transformation (e.g., by a DFT) the actualsub-band vector (at time n) may be obtained. The DFT may be formulatedby the transformation matrix:

$T = {\begin{bmatrix}1 & 1 & 1 & \ldots & 1 \\1 & {\mathbb{e}}^{{- j}\;\frac{1}{M}2\pi} & {\mathbb{e}}^{{- j}\;\frac{2}{M}2\pi} & \ldots & {\mathbb{e}}^{{- j}\;\frac{M - 1}{M}2\pi} \\1 & {\mathbb{e}}^{{- j}\;\frac{2}{M}2\pi} & {\mathbb{e}}^{{- j}\;\frac{4}{M}2\pi} & \ldots & {\mathbb{e}}^{{- j}\;\frac{2{({M - 1})}}{M}2\pi} \\\vdots & \vdots & \vdots & \ddots & \vdots \\1 & {\mathbb{e}}^{{- j}\;\frac{M - 1}{M}2\pi} & {\mathbb{e}}^{{- j}\;\frac{2{({M - 1})}}{M}2\pi} & \ldots & {\mathbb{e}}^{{- j}\;\frac{{({M - 1})}{({M - 1})}}{M}2\pi}\end{bmatrix}.}$

The sub-band signal may be down-sampled by the factor r and, thus, thedown-sampled sub-band signal at time n may be obtained by:y _(sb)(n)=TF ₀ y(nr).

By means of the respective window matrices for the previous (n−1) andsubsequent (n+1) sub-band vectors:F ⁻¹ =[F0_(M×2r)] and F ₁=└0_(M×2r) F┘,the following signal vectors may be obtained:y _(sb)(n−1)=TF ⁻¹ y(nr) and y _(sb)(n+1)=TF ₁ y(nr).

In order to extract odd sub-band vectors only the matrix:

$E_{u} = \begin{bmatrix}0 & 1 & 0 & 0 & 0 & 0 & \ldots & 0 & 0 \\0 & 0 & 0 & 1 & 0 & 0 & \ldots & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 1 & \ldots & 0 & 0 \\\vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\0 & 0 & 0 & 0 & 0 & 0 & \ldots & 0 & 1\end{bmatrix}$is defined to obtain sub-bands for odd sub-band indices:y _(sb,u)(n)=E _(u) y _(th)(n)=E _(u) TF ₀ y(nr).

Similarly, extraction of sub-band signals with even indices results fromy _(sb,g)(n−i)=E _(g) y _(sb)(n−i)=E _(g) TF _(−i) y(nr);i=±1,with the extraction matrix:

$E_{g} = {\begin{bmatrix}1 & 0 & 0 & 0 & 0 & 0 & \ldots & 0 & 0 \\0 & 0 & 1 & 0 & 0 & 0 & \ldots & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 1 & \ldots & 0 & 0 \\\vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\0 & 0 & 0 & 0 & 0 & 0 & \ldots & 1 & 0\end{bmatrix}.}$

Reconstruction of the odd sub-band vectors may be achieved byinterpolation of even sub-band vectors. At time n a reconstructed oddsub-band vector ŷ_(sb,u)(n) may be calculated from an actual (time is n)even sub-band signal vector y_(sb,g)(n) and a previous (time is n−1) anda sub-sequent (time is n+1) even sub-band vector:ŷ _(sb,u)(n)=C ₁ y _(sb,g)(n+1)+C ₀ y _(sb,g)(n)+C ⁻¹ y _(sb,g)(n−1)with the interpolation matrices C₁, C₀, C⁻¹. In principle, averaging bymore than two adjacent (in time) signal vectors may be performed, e.g.,ŷ _(sb,u)(n)=C ₂ y _(sb,g)(n+2)+C ₁ y _(sb,g)(n+1)+C ⁻¹ y _(sb,g)(n−1)+C⁻² y _(sb,g)(n−2).

With the above expression for sub-band signals with even indices onegets for the interpolation with C₁, C₀, and C⁻¹:

$\begin{matrix}{{{\hat{y}}_{{sb},u}(n)} = {{C_{1}E_{g}{TF}_{1}{y({nr})}} + {C_{0}E_{g}{TF}_{0}{y({nr})}} + {C_{- 1}E_{g}{TF}_{- 1}{y({nr})}}}} \\{= {\left\lbrack {{C_{1}E_{g}{TF}_{1}} + {C_{0}E_{g}{TF}_{0}} + {C_{- 1}E_{g}{TF}_{- 1}}} \right\rbrack{y({nr})}}}\end{matrix}$This expression may be represented by:ŷ _(sb,u)(n)=C _(ges) D _(E) _(g) _(T) F _(ges) y(nr)with C_(ges)=[C₁C₀C⁻¹] and the block diagonal matrix:

$D_{E_{g}T} = \begin{bmatrix}{E_{g}T} & 0 & 0 \\0 & {E_{g}T} & 0 \\0 & 0 & {E_{g}T}\end{bmatrix}$and the total window matrix

$F_{ges} = {\begin{bmatrix}F_{1} \\F_{0} \\F_{- 1}\end{bmatrix}.}$

In one implementation, to find a reconstruction for the previouslyexcised sub-band vector, the L₂-norm of the difference vector may beminimized:∥y _(sb,u)(n)−ŷ _(sb,u)(n)∥₂, i.e., ∥E _(u) TF ₀ y(nr)−C _(ges) D _(E)_(g) _(T) F _(ges) y(nr)∥₂.The minimization may be achieved in a sufficiently good approximation bydetermining C_(ges) such that each row of the matrixE_(u)TF₀−C_(ges)D_(E) _(g) _(T)F_(ges) has a minimal L₂-norm.This may be achieved by the Moore-Penrose-pseudo inverse:(D _(E) _(g) _(T) F _(ges))′ of the matrix (D _(E) _(g) _(T) F _(ges)).

Thus, C_(ges) may be expressed by C_(ges)=E_(u)TF₀(D_(E) _(g) _(T)F_(ges)), under the assumption that (D_(E) _(g) _(T) F_(ges)) (D_(E)_(g) _(T) F_(ges))^(H) is invertable (where the upper index H denotesthe Hemite conjugate, i.e., the adjoint matrix). TheMoore-Penrose-pseudo inverse may be calculated from:(D _(E) _(g) _(T) F _(ges))′=(D _(E) _(g) _(T) F _(ges))^(H)[(D _(E)_(g) _(T) F _(ges))(D _(E) _(g) _(T) F _(ges))^(H)]⁻¹.

A direct use of the interpolation matrices C⁻¹ and C₁ may demand highcomputer capacities. Therefore, approximates of these matrices C⁻¹ andC₁ by their respective main and secondary diagonals may be used.

The output signals after echo compensation and/or other processing fornoise reduction, dereverberation, etc., for the sub-bands that are notexcised may be denoted by ŝ_(μ)(n), where μ is the sub-band index.ŝ_(μ)(n) may be determined for all sub-bands (including thereconstruction of the previously excised sub-band vectors) by thefollowing equation:

${{\overset{\sim}{s}}_{\mu}(n)} = \left\{ \begin{matrix}\begin{matrix}{{\sum\limits_{k = {- 1}}^{1}{{{\hat{s}}_{\mu - 1}\left( {n - k} \right)}{C_{k}\left( {\mu,\mu} \right)}}} +} \\{{\sum\limits_{k = {- 1}}^{1}{{{\hat{s}}_{\mu + 1}\left( {n - k} \right)}{C_{k}\left( {\mu,{\mu + 1}} \right)}}},{{{if}\mspace{14mu}{{mod}\left( {\mu,2} \right)}} = 1}}\end{matrix} \\{{{\hat{s}}_{\mu}\left( {n - 1} \right)},{else}}\end{matrix} \right.$where C_(k)(n₁, n₂) denotes the element in the n₁ ^(th) row and the n₂^(th) column of the matrix C_(k). This implies that even sub-bands aretaken with a delay of one time increment (n−1).

Reconstruction of a previously excised sub-band signal may be based onmore than one preceding and subsequent sub-band signal (n−1 and n+2). Inparticular, a different number of preceding and subsequent sub-bandsignals may be used for the interpolation (C₀≠0).

A set (e.g., a complete set) of sub-band signals {tilde over(s)}_(sb)(n) may be input into a synthesis filter bank 218 to synthesizethe enhanced sub-band signals ŝ_(sb,g) (n) with the reconstructedsub-band signals. The synthesis filter bank 218 may correspond to theanalysis filter bank 204 used to divide the detected signal y(n) intothe audio sub-band signals y_(sb)(n). The synthesis filter bank 218combines the enhanced sub-band signals with the reconstructed sub-bandsignals to form a full-band enhanced microphone signal {tilde over(s)}(n). The synthesis filter bank 218 may include a Hann or Hammingwindow. The analysis filter banks 204 and 210 may down-sample thesub-band signals by a factor r. The synthesis filter bank 218 mayup-sample the down-sampled reconstructed and enhanced microphonesub-band signals ŝ_(sb,g) (n) by the same factor as the down-samplingfactor r. The synthesis filter bank 218 may be represented by:g _(μ,syn) =└g _(μ,0,syn) , . . . ,g _(μ,N) _(syn) _(-1,syn)┘^(T)

For one application including M=256 sub-bands and down-sampling rates ofr=64 and r=72, for example, computational time and memory demand may bereduced by about 50% as compared to standard DFT processing. The timefor signal processing (delay time) may only be a few milliseconds abovethe time delay of standard processing by means of polyphase filterbanks. Also, the delay time may be below the threshold according to theGlobal System for Mobile communications (GSM) standards of 39 ms invehicle cabins. Moreover, the adaptation velocity of the echocompensation filter 208 may only be slightly different standardprocessing.

FIG. 3 is a process that enhances an audio signal by processing thesignal in the sub-band regime. At act 302, an audio signal is obtained.The audio signal may be a speech signal representing an utterance by alocal speaker. A microphone or other detection device may detect theaudio signal. Alternatively, the audio signal may be detected by amicrophone array to obtain a number of microphone signals that may byprocessed by beamforming. In this case, the signal processing describedin the following acts may be applied to each of the microphone signalsobtained by the microphones of the microphone array. At act 304, theaudio signal is divided into audio sub-band signals. Some or all ofthese audio sub-band signals may be subsequently processed to enhancethe quality.

After dividing the audio signal into audio sub-band signals, a portionof the audio sub-band signals may be excised at act 306. In oneimplementation, all microphone sub-band signals y_(μ) with an odd indexmay be excised and only microphone sub-band signals y_(μ) with an evenindex με{0, 2, 4, . . . , M−2} may be maintained. In anotherimplementation, all microphone sub-band signals y_(μ) with an even indexmay be to excised and only microphone sub-band signals y_(μ) with an oddindex με{1, 3, 5, . . . , M−1} may be maintained. By excising about halfof the audio sub-band signals, the processing and memory demands mayaccordingly be reduced by about half.

In another implementation, the audio sub-band signals are selectivelyexcised. The system may excise audio sub-band signals above or below apredetermined frequency threshold. In particular, selected audiosub-band signals may be excised in one frequency range of the sub-bandsignals, while the sub-band signals in other frequency ranges are notexcised or are excised to a lesser degree. In another implementation, agreater percentage of the audio sub-band signals that are above or belowa predetermined frequency threshold may be excised. For example, a firstpercentage of audio sub-band signals may be excised in a first frequencyrange, while a second percentage of audio sub-band signals may beexcised in a second frequency range. The first percentage may be thesame or different than the second percentage. Also, the percentages areadjustable and may be as high as 100% or as low as 0%.

In one implementation, a predetermined number of audio sub-band signalsmay be excised only from a high frequency range (e.g., above somethreshold, such as above 1 kHz, 1.5 kHz, or 2 kHz) while keeping all (orsubstantially all) of the audio sub-band signals in a lower frequencyrange. Thereby, a variety of compromises between saving computationalcosts and achieving high signal quality may be achieved.

At act 308, the remaining audio sub-band signals may be enhanced. Theremaining sub-band signals may be processed for echo compensation,dereverberation, noise reduction, and/or another signal enhancementtechnique. At act 310, at least a portion of the previously excisedsub-band signals are reconstructed. In one implementation, excised audiosub-band signals may be reconstructed from the remaining audio sub-bandsignals. At act 312, the reconstructed sub-band signals are synthesizedwith the enhanced sub-band signals to generate a full-band enhancedaudio signal.

FIG. 4 is a process that uses a reference signal to enhance an audiosignal. At act 402, a reference signal is obtained. In oneimplementation, the reference signal represents a noise component thatmay exist in a detected signal. In another implementation, the referencesignal is a signal that represents possible echo components that mayexist in a detected signal. Specifically, the reference signal may be anaudio signal that is transmitted from a system loudspeaker and may bedetected by a system microphone.

At act 404, the reference signal is divided into reference sub-bandsignals. After dividing the reference signal into reference sub-bandsignals, a portion of the reference sub-band signals may be excised. Inone implementation, the reference sub-band signals may be excised to thesame degree as the audio sub-band signals. At act 406, it is determinedwhich of the audio sub-band signals were excised at act 306 of FIG. 3.In one implementation, the microphone sub-band signals with an odd indexnumber were excised and the microphone sub-band signals with an evenindex number were maintained. In another implementation, the microphonesub-band signals with an even index number were excised and themicrophone sub-band signals with an odd index number were maintained. Inyet another implementation, a different excising scheme was implemented.

At act 408, a subset of the reference sub-band signals are excised. Theexcised subset may correspond to the subset of the audio sub-bandsignals that were excised. If the audio sub-band signals with odd indexnumbers were excised, then the reference sub-band signals with odd indexnumbers may also be excised at act 408. Therefore, the remaining subsetof the reference sub-band signals may correspond to the remaining subsetof the audio sub-band signals.

At act 410, the remaining reference sub-band signals are used to enhancethe remaining audio sub-band signals. In one implementation, theremaining reference sub-band signals may represent a noise or echocomponent that may be present in the remaining audio sub-band signals.Therefore, the remaining reference sub-band signals may be used toattenuate the noise or echo components present in the remaining audiosub-band signals. In one implementation, the filter coefficients of anecho compensation filter may be adapted based on the remaining referencesub-band signals and sub-band error signals. The remaining audiosub-band signals may then be filtered by the adapted filter coefficientsto reduce echo contributions in the remaining audio sub-band signals.Specifically, estimated echo contributions may be subtracted from theremaining audio sub-band signals.

FIG. 5 is a process that reconstructs excised sub-band signals. At act502, enhanced audio sub-band signals are received. The enhanced audiosub-band signals 113 may be enhanced versions of the audio sub-bandsignals that were not excised at act 306 of FIG. 3. At act 504, theexcised sub-band signals are identified. For example, locations ofexcised sub-band signals may be identified within the full-bandspectrum. At act 506, one or more of the remaining sub-band signals areselected for use to reconstruct the excised sub-band signals.

At act 508, the excised sub-band signal is reconstructed based on theselected remaining sub-band signals. An excised sub-band signal may bereconstructed by averaging multiple remaining audio sub-band signals.The reconstruction generates an audio sub-band signal to replace theidentified excised audio sub-band signal. The excised sub-band signalsmay be reconstructed from one, two, or more remaining sub-band signals.In one implementation, an excised sub-band signal may be reconstructedby averaging remaining audio sub-band signals that are adjacent in timeto the excised audio sub-band signal. In another implementation, anexcised audio sub-band signal from a particular time may bereconstructed by interpolation of remaining audio sub-band signals fromthe particular time and remaining audio sub-band signals that areadjacent in time.

The reconstruction may be a weighted average of multiple remainingsub-band signals. In an implementation where two sub-band signals areaveraged to reconstruct an excised sub-band signal, a first weightingfactor may be applied to the first sub-band signal and a secondweighting factor may be applied to the second sub-band signal. Therelative weights of the multiple sub-band signals may be controlled oradjusted to reconstruct the excised sub-band signals.

In one implementation, at least at one time (n) the audio sub-bandsignals for the predetermined sub-bands for which audio sub-band signalswere excised may be reconstructed for the predetermined sub-bands byaveraging the remaining audio sub-band signals that are adjacent in time(n+k, n−k), where n is the discrete time index and k is an integer, k≧1;k=1, 2, etc. Thus, the term adjacent may include the closest adjacentsignals (in time) as well as some number of neighbors. A reconstructedaudio sub-band signal at frequency bin j may be calculated by averagingenhanced (e.g., echo/noise compensated) remaining audio sub-band signalsat frequency bins j+1 and j−1. If a predetermined number of microphonesub-band signals are excised all over the set of sub-bands (μ=1, . . . ,M) averaging may be performed also all over the entire range ofsub-bands. Alternatively, it may be preferred to reconstruct a part ofthe excised microphone sub-band signals only. Reconstruction may bevariably performed according to the actual application. Averaging mayinclude different weights (interpolation matrices) for the audiosub-band signals at times n+1, n and n−1 (and further adjacent values,when used).

In another implementation, the excised microphone sub-band signals attime n may be reconstructed by interpolation of remaining microphonesub-band signals at the time n and remaining microphone sub-band signalsadjacent in time (e.g., one or more previous signal vectors and/orsubsequent signal vectors). Accurate reconstruction with tolerableartifacts may thereby be achieved. To achieve a significant reduction ofthe need for computational resources, the interpolation may be performedby interpolation matrices which are approximated by their main diagonalsand secondary diagonals, respectively.

FIG. 6 is a process that compensates for echo in a microphone signal. Atact 602, a verbal utterance is detected by a microphone. The microphonethen generates a microphone signal that represents the verbal utterance.At act 604, microphone sub-band signals are obtained. Specifically, themicrophone signal may be divided into multiple microphone sub-bandsignals. At act 606, a predetermined number of the microphone sub-bandsignals are excised.

At act 608, the remaining microphone sub-band signals are echocompensated. Specifically, the echo compensation may attempt toattenuate echo components in the remaining microphone sub-band signals.Echo compensated microphone sub-band signals may be further processedfor noise reduction. Moreover, the microphone sub-band signals may bede-correlated by a time-invariant de-correlation filter (e.g., a filterof the first or second order) or by an adaptive de-correlation means inorder to improve the convergence speed of the adaptation process of thefilter coefficients of the echo compensation filter.

At act 610, the excised sub-band signals may be reconstructed based onthe echo compensated sub-band signals. At act 612, the sub-band signalsare synthesized to obtain an enhanced microphone signal. The echocompensated sub-band signals may be combined with the reconstructedsub-band signals to obtain the full-band enhanced microphone signal.

Each of the processes described may be encoded in a computer readablemedium such as a memory, programmed within a device such as one or morecircuits, one or more processors or may be processed by a controller ora computer. If the processes are performed by software, the software mayreside in a memory resident to or interfaced to a storage device, acommunication interface, or non-volatile or volatile memory incommunication with a transmitter. The memory may include an orderedlisting of executable instructions for implementing logic. Logic or anysystem element described may be implemented through optic circuitry,digital circuitry, through source code, through analog circuitry, orthrough an analog source, such as through an electrical, audio, or videosignal. The software may be embodied in any computer-readable orsignal-bearing medium, for use by, or in connection with an instructionexecutable system, apparatus, or device. Such a system may include acomputer-based system, a processor-containing system, or another systemthat may selectively fetch instructions from an instruction executablesystem, apparatus, or device that may also execute instructions.

A computer-readable medium, machine-readable storage medium,propagated-signal medium, and/or signal-bearing medium may comprise anydevice that contains, stores, communicates, propagates, or transportssoftware for use by or in connection with an instruction executablesystem, apparatus, or device. The machine-readable medium mayselectively be, but not limited to, an electronic, magnetic, optical,electromagnetic, infrared, or semiconductor system, apparatus, device,or propagation medium. A non-exhaustive list of examples of amachine-readable medium would include: an electrical connection havingone or more wires, a portable magnetic or optical disk, a volatilememory such as a Random Access Memory “RAM,” a Read-Only Memory “ROM,”an Erasable Programmable Read-Only Memory (EPROM or Flash memory), or anoptical fiber. A machine-readable medium may also include a tangiblemedium upon which software is printed, as the software may beelectronically stored as an image or in another format (e.g., through anoptical scan), then compiled, and/or interpreted or otherwise processed.The processed medium may be stored in a computer and/or machine memory.

While various embodiments of the invention have been described, it willbe apparent to those of ordinary skill in the art that many moreembodiments and implementations are possible within the scope of theinvention. Accordingly, the invention is not to be restricted except inlight of the attached claims and their equivalents.

We claim:
 1. A method for audio signal processing, comprising: dividinga microphone signal into microphone sub-band signals; excising apredetermined number of the microphone sub-band signals forpredetermined sub-bands; processing the remaining microphone sub-bandsignals by attenuating noise or echo components in the remainingmicrophone sub-band signals to obtain enhanced microphone sub-bandsignals; and reconstructing microphone sub-band signals for thepredetermined sub-bands for which microphone sub-band signals wereexcised, where each of the excised microphone sub-band signals isreconstructed from the enhanced microphone sub-band signals obtained byprocessing the remaining microphone sub-band signals; whereinattenuating the noise or echo components comprises: dividing a referencesignal into reference sub-band signals: excising a predetermined numberof the reference sub-band signals that is equal to the predeterminednumber of excised microphone sub-band signals for the same predeterminedsub-bands: adapting filter coefficients of an echo compensation filterbased on the remaining reference sub-band signals: and filtering theremaining microphone sub-band signals with the adapted filtercoefficients.
 2. The method of claim 1, wherein the microphone sub-bandsignals and the reference sub-band signals are down-sampled with respectto the microphone signal and the reference signal, respectively, by thesame down-sampling factor.
 3. The method of claim 1, wherein the act ofexcising the predetermined number of the microphone sub-band signalscomprises: excising each of the microphone sub-band signals with an oddindex number and maintaining each of the microphone sub-band signalswith an even index number; or excising each of the microphone sub-bandsignals with an even index number and maintaining each of the microphonesub-band signals with an odd index number.
 4. The method of claim 1,wherein the act of excising the predetermined number of the microphonesub-band signals comprises excising a greater percentage of themicrophone sub-band signals that are above or below a predeterminedfrequency threshold.
 5. A method of for audio signal processing,comprising: dividing a microphone signal into microphone sub-bandsignals; excising a predetermined number of the microphone sub-bandsignals for predetermined sub-bands: processing the remaining microphonesub-band signals to obtain enhanced microphone sub-band signals; andreconstructing microphone sub-band signals for the predeterminedsub-bands for which microphone sub-band signals were excised, where eachof the excised microphone sub-band signals is reconstructed from theenhanced microphone sub-band signals obtained by processing theremaining microphone sub-band signals: wherein the act of reconstructingthe microphone sub-band signals comprises reconstructing an excisedmicrophone sub-band signal by averaging remaining microphone sub-bandsignals that are adjacent in time to the excised microphone sub-bandsignal.
 6. A method for audio signal processing, comprising: dividing amicrophone signal into microphone sub-band signals; excising apredetermined number of the microphone sub-band signals forpredetermined sub-bands; processing the remaining microphone sub-bandsignals to obtain enhanced microphone sub-band signals; andreconstructing microphone sub-band signals for the predeterminedsub-bands for which microphone sub-band signals were excised, where eachof the excised microphone sub-band signals is reconstructed from theenhanced microphone sub-band signals obtained by processing theremaining microphone sub-band signals; wherein the act of reconstructingthe microphone sub-band signals comprises reconstructing excisedmicrophone sub-band signals from a particular time by interpolation ofremaining microphone sub-band signals from the particular time andremaining microphone sub-band signals that are adjacent in time.
 7. Themethod of claim 6, wherein the interpolation is performed byinterpolation matrices that are approximated by their main diagonals andsecondary diagonals, respectively.
 8. A signal processing system,comprising: an analysis filter bank configured to divide an audio signalinto audio sub-band. signals; a first filter configured to excise asubset of the audio sub-band signals; a second filter configured toprocess a remaining subset of the audio sub-band signals to obtainenhanced audio sub-band signals; a processor configured to reconstructat least a portion of the subset of the audio sub-band signals that wereexcised; and a synthesis filter bank configured to synthesize thereconstructed audio sub-band signals with the enhanced audio sub-bandsignals to form an enhanced audio signal; wherein the processor isconfigured to average a first signal of the enhanced audio sub-bandsignals and a second signal of the enhanced audio sub-band signals togenerate an audio sub-band signal to replace one of the excised audiosub-band signals.
 9. The system of claim 8, wherein the processor isconfigured to reconstruct the excised subset of the audio sub-bandsignals from the remaining subset of the audio sub-band signals.
 10. Thesignal processing system of claim 8, further comprising: a microphoneconfigured to detect the audio signal and pass the audio signal to theanalysis filter bank.
 11. The system of claim 8, further comprising: apost-filter configured to filter the enhanced audio sub-band signals toreduce background noise or residual echoes.
 12. A signal processingsystem comprising: an analysis filter bank configured to divide an audiosignal into audio sub-band signals; a first filter configured to excisea subset of the audio sub-band signals; a second filter configured toprocess a remaining subset of the audio sub-band signals to obtainenhanced audio sub-band signals; a processor configured to reconstructat least a portion of the bset of the audio sub-band signals that wereexcised; and a synthesis filter bank configured to synthesize thereconstructed audio sub-band signals with the enhanced audio sub-bandsignals to form an enhanced audio signal; wherein the processor isconfigured to reconstruct excised audio sub-band signals from aparticular time by interpolation of remaining audio sub-band signalsfrom the particular time and remaining audio sub-band signals that areadjacent in time.
 13. A signal processing system comprising: an analysisfilter bank configured to divide an audio signal into audio sub-bandsignals; a first filter configured to excise a subset of the audiosub-band signals; a second filter configured to process a remainingsubset of the audio sub-band signals to obtain enhanced audio sub-bandsignals; a processor configured to reconstruct at least a portion of thesubset of the audio sub-band signals that were excised; and a synthesisfilter bank configured to synthesize the reconstructed audio sub-bandsignals with the enhanced audio sub-band signals to form an enhancedaudio signal; wherein the second filter comprises an echo compensationfilter, the system further comprising: an analysis filter bankconfigured to divide a reference signal into reference sub-band signals;a third filter configured to excise a subset of the reference sub-bandsignals that is equal in number to the excised subset of the audiosub-band signals; and where the echo compensation filter is configuredto be adapted based on a remaining subset of the reference sub-bandsignals, where the echo compensation filter is configured to use adaptedfilter coefficients to remove echo components from at least a portion ofthe remaining subset of the audio sub-band signals.